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A BSTRACT 


A review  is  presented  of  the  project  to  develop  and  field  test  the  ARPA 
Authoring  System  for  use  by  subject-matter  experts  in  the  preparation 
of  task  training  in  on-the-job  environments.  Materials  were  prepared 
according  to  the  ARPA  Authoring  System  procedures,  and  both  prepara- 
tion and  delivery  were  on  the  facilities  of  the  Lincoln  Terminal  System. 
An  analysis  of  the  lesson  production  rates  for  30  lessons  prepared  by 
Air  Force  personnel  revealed  no  systematic  effects  related  to  individual 
author,  authoring  experience,  or  lesson  type.  Preliminary  results  from 
a field  test  of  the  materials  indicate  that  the  lessons  are  sound.  Large 
savings  were  found  in  the  time  trainees  require  to  reach  job  qualification 
and  in  the  amount  of  labor  that  subject-matter  experts  must  dedicate  to 
training  duties. 
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1.  lNTHOlH’CTlON 

This  is  tile  final  Semiannual  Technical  Summary  rt‘|xni  of  this  development  project.  The 
primary  goal  was  to  develop  a system  for  subject-matter  experts  to  author  materials  in  the  on- 
the-job  environment,  The  area  of  applieation  selected  was  task  training,  the  kind  of  instruction 
provided  at  the  work  site  in  one-on-one  tutorial  mode  to  bridge  the  gap  between  formal  technical 
schooling  and  actual  readiness  to  perform  work,  l'he  immediate  goal  was  to  demonstrate  that 
expert  technicians  could  prepare  training  materials  for  delivery  on  a computer-based  delivers 
system.  The  intent  is  to  capture  training  m lessons  when  the  expert  technicians  are  available, 
so  that  training  can  be  conducted  at  times  when  the  experts  are  preoccupied.  The  project  in- 
cluded development  of  the  authoring  sj  stem,  recording  and  analysts  of  lesson  production  costs, 
test  of  the  lessons  produced,  and  assessment  of  the  impact  of  this  form  of  instruction  on  per- 
formance training  in  on-the-job  environments. 

The  project  was  a joint  venture.  The  Air  Force  Communications  Services  (AFCS1  Com- 
mand provided  subject-matter  experts  as  authors  and  a work  setting  within  which  to  test  the 
lesson  products.  Lincoln  Laboratory  constructed  the  hardware  elements  to  sup|>ort  the  tests, 
a Lincoln  Terminal  System  (LTS)*  consisting  of  five  l l'S-s  instructional  delivery  units  amt  one 
ALII  (Audio  Logic  Weoorderl  a small  computerized  facility  on  which  to  make  audio  recordings, 
to  program  lessons,  and  to  analyze  student  records.  Facilities  were  also  provided  for  conver- 
sion of  visual  materials  and  tape  recordings  to  microfiche  (film  cards)  suitable  for  the  1 l'S-s. 

The  LTS  that  combines  micrographic  and  microprocessor  technology  was  developed  espe- 
cially as  a technology -based  system  for  on-the-job  training.  There  are  two  principal  features 
of  the  LTS  that  distinguish  it  from  other  computer-based  means  of  instruction:  (a)  The  1.  l'S-s 
operates  stand-alone,  connected  only  to  a source  of  electric  power;  and  (hi  the  System  depends 
on  conventional  techniques  for  the  design  of  lesson  materials.  The  delivery  of  instruction  is 
substantially  automated,  and  the  preparation  of  materials  can  be  done  by  the  usual  training  per- 
sonnel. The  effectiveness  of  LTS  training  has  been  demonstrated  previously  on  subjects  such 
as  basic  electronics  for  Air  Force  technicians,  electrical  practice  for  coal  miners,  and 
digital -systems  engineering  for  graduate  engineers.1 

The  project  has  provided  an  op|«ortunity  to  study  the  means  for  generating  computer-based 
instruction  under  unusual  circumstances: 

• Subject-matter  experts  as  authors  (not  educational  specialists) 

• A procedural  approach  to  managing  the  authoring  process  (as  contrasted 
to  training  the  authors  in  educational  techniques) 

• Computer-based  instruction  at  scattered  locations  in  (In'  on -the -job 
environment  (not  in  a school  setting) 

• Technology-based  work  performance  training  (not  labor  intensive 
tutoring). 

In  addition,  the  Air  Force  has  been  able  to  test  a new  solution  to  a persistent  training  problem 
the  acute  shortage  of  experienced  technicians  to  conduct  training  at  the  work  site.  It  has  per- 
mitted Lincoln  Laboratory  to  complete  implementation  of  the  LTS  by  providing  authoring 
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procedures  m a critical  applications  area.  finally,  the  project  lias  served  as  a test  bed  for  the 
new,  prototype  hardware  for  the  Lincoln  Terminal  System  developed  bi  the  V’.S.  Bureau  of  Mines. 

II.  THE  PROJECT 

A.  Activities 

The  project7  **'  was  carried  out  in  three  phases. 

Phase  I:  March  t 7«-  October  t97(>.  The  first  version  of  the  authoring 
procedures  was  prepared.  Two  LTS-5  units  were  constructed  for  tests 
of  lessons. 

Phase  October  t 978  - August  1977.  The  authoring  procedures  were 
used  by  technical  sergeants,  and  the  lessons  produced  were  tested  oil 
novice  technicians  on  a limited  scale;  the  authoring  procedures**’*^ 
were  revised  on  the  basis  of  results.  Management  procedures  ‘ for 
use  at  the  unit  level  to  direct  the  lesson-development  efforts  were  pre- 
pared. Plans  were  developed  for  a full-scale  operational  test  and  evalu- 
ation of  the  authoring  and  instructional  delivery  systems  in  KY  78. 

Phase  1:  September  1977  September  1978.  Thirty  lessons  on  the 
AN/TRC-97A  radio  set  were  prepared  by  Air  force  technical  per- 
sonnel as  finished  materials.  The  lessons  were  converted  to  micro- 
fiche by  Lincoln  personnel.  Data  on  the  authoring  effort  were  gathered 
for  all  lessons  anil  analyzed.  Assistance  was  given  to  the  Command  to 
establish  an  evaluation  of  the  cost  and  effectiveness  of  the  training  by 
an  independent  research  organization.  Preliminary  results  of  the 
training  were  analyzed,  and  the  implications  of  the  findings  are  re- 
viewed below. 

It.  Participants 

The  major  participants  were  Lincoln  Laboratory  - who  provided  technical  services,  pro- 
cedures. and  facilities  to  support  authoring  and  training  and  the  Tactical  Communications 
Area  Command  within  the  API'S  who  provided  the  field  environment  and  resources  necessary 
to  test  the  new  technology.  Lincoln  participation  was  funded  largely  within  the  ARPA  Authoring 
System  budget,  and  the  API'S  contribution  was  in  the  form  of  personnel  and  facilities.  Additional 
funds  were  obtained  from  the  PHAM  office  of  the  Air  force  Logistics  Command.  PRAM  is  a 
special  program  to  assist  Operational  Commands  in  the  field  testing  of  new  technology  for  which 
near-term  cost  benefits  are  apparent.  The  PRAM  money  provided  for  more  instructional  units 
and  for  the  independent  evaluation  of  the  effort,  upgrading  the  field  trial  to  one  representative 
of  a full-scale  initial  operational  test  and  evaluation.  The  Human  Resources  Laboratory  of  the 
Air  force  Systems  Command  assisted  in  establishing  the  evaluation  study. 

III.  AUTHORING  SYSTEM  EVALUATION 

A.  Lesson  Preparation 

The  area  of  application  chosen  for  the  test  of  the  authoring  procedures  was  training  on  per- 
formance of  maintenance  procedures  on  the  AN/THC-97A  radio  set.  a large  transniitter/receiver 
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located  in  .1  mobile  iomimum.it ions  van.  I'lurti  lessons  were  prepared  by  airmen  anil  trdr 
meal  sergeants  on  tin-  topu  s hstoil  m Cable  l.  l'he  leail  renter  for  the  authoring  effort  was  the 
Mh  LVGp  (Combat  Communications  Group)  within  I'acttcul  lommuimiitiims  Area  of  the  Al'C'S. 
at  liobtns  Air  Force  Itase,  Georgia. 

l'he  principal  imlepemlent  variables  nlentifteil  as  of  interest  relative  to  the  authoring  pro- 
ve ss  were 

• Author  principal  person  implementing  a lesson 

• Preparation  Sequence  first-vs-secoiui  effort  by  an  author 

• l.esson  l'ype  task -with -explanation  vs  pure  procedure. 


TABU 

LESSONS  PREPARtD  FOR  THE 

1 

AN,  TRC-97A  RADIO  SET 

Categoty 

Title 

1.  Performance  Checks  ! 1. 

Shelter  and  Equipment  Turn-On 

2. 

Normal  and  Emergency  Turn-Off 

3. 

Power  Amplifier  Turn-On 

4. 

Power  Amplifier  Turn-Off 

5. 

Multiplexer  loop  Performance  Check 

t>. 

Synthesizer  Frequency  and  Power  Check 

7. 

Excifei  Performance  Check 

8. 

RF  Loop  Performance  Check 

9. 

Receiver  (FM^  Quieting  Performance  Check 

2.  Adjustments  and  Alignments  1. 

Multiplexer  Voltage  Regulator 

and  Master  Oscillator  Alignment 

2. 

A 7 Test  Set  Operation  and  Alignment 

3. 

Multiplexer  Transmit  Path  Alignment 

4. 

Multiplexer  Receive  Path  Alignment 

5. 

Multiplexer  Alarm  Alignment 

6. 

Multiplexer  Ring  Window  Alignment 

7. 

Modulator  Alignment 

8. 

Threshold  Extender  Adjustment 

9. 

Signal  Comparator  Alignment 

10. 

Radio  Net  Gain  Adjustment 

11. 

RF  Power  Monitor  Meter  Calibration 

12. 

Power  Amplifier  Low  Powei  Alarm 

and  A24  Monitor  Alignment 

13. 

Teletype  Adjustment 

14. 

A21  Power  Supply  Alignment 

3.  Miscellaneous 

None 

4.  Operations  1, 

Operation  of  the  Pocket  Transit 

2. 

Remote  Alarm  Monitor  lBZ-1091 

3. 

Jamming  and  ECM 

4. 

Operation  of  the  Control  Monitor 

5. 

Operation  of  Test  Equipment 

6. 

Site  Installation 

7. 

Van  Orientation 

TABLE  II 


CONDITIONS  OF  AUTHORING  AND  MEASURES  OF  PERFORMANCE 
IN  THE  PRODUCTION  OF  30  LESSONS  FOR  THE  AN  TRC-*7A  RADIO  SET 


Two  measures  of  lesson -production  el'fieiene>  were  recorded:  man-hours  of  effort  for  each 
lesson,  ami  calendar  day  s between  start  and  finish.  The  first  is  a measure  of  cost,  and  the 
second  an  indication  of  the  res|>onsiveness  of  the  authoring  system  to  new  training  demands. 

Table  11  shows  the  duta  for  the  '0  lessons,  each  characterized  by  the  Author.  Preparation 
Sequence,  and  l.esson  Type  variables.  To  define  the  l.esson  Type,  each  of  the  frames  was 
examined  for  content  in  terms  of  "task"  and  "explanatory  " information  categories  as  defined  in 
the  authoring  procedures.  The  measure  is  the  ratio  of  tin-  number  of  frames  that  contain  task 
information  to  the  sum  of  those  that  contain  task  information  amt  those  that  contain  explanatory 
information.  Thus,  a lesson  with  purely  procedural  content  (task  instructions)  is  100  percent 
on  the  scale,  and  one  that  is  purely  explanatory  (concepts,  facts,  etc.)  is  0 percent.  The  dis- 
tribution of  lessons  by  Lesson  Ty  pe  is  shown  in  T ig.  1.  The  datum  at  the  left,  for  the  lesson 
Van  Orientation,  is  only  1 -percent  procedural  and  therefore  is  not  a valid  example  of  a product 
of  authoring  procedures  which  were  designed  specifically  for  task-oriented  instruction.  Other- 
wise. the  lessons  are  highly  procedural,  few  of  them  are  as  low  as  s0  percent  on  the  scale.  This 
is  consistent  with  the  objective  stated  in  the  Lesson  Specification  in  every  case,  to  prepare  the 
novice  to  perform  a maintenance  procedure  accurately  and  safely.  As  a consequence,  related 
facts,  concepts,  precautions,  etc.  were  limited  in  scope  and  were  usually  associated  with  single 
steps  and  not  with  whole  parts  or  the  entire  procedure.  (Much  of  the  conceptual  material  the 
system  block  diagram,  equipment  layout,  and  circuits  was  covered  in  prior,  formal  instruction. ) 
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Fig.  t.  Frequency  distribution  of  Lesson  Type  measure  for  '0  lessons. 

The  measures  of  performance  are  shown  for  each  lesson  in  Table  II.  These  include  the 
number  of  frames,  the  calendar  days  to  complete  a lesson,  and  the  man-hours  expended  by  the 
principal  author  and  the  support  personnel  for  each  lesson.  Time-per-frame.  the  total  man- 
hours divided  by  the  number  of  frames,  was  calculated  for  each  lesson,  a measure  that  is  in- 
tended to  take  out  the  effect  of  the  sire  of  a lesson  as  a determinant.  A summary  of  these  data 

* 

shows  that,  on  the  average.  -88  man-hours  spread  over  7s  calendar  days  is  required  to  produce 
a lesson. 

An  analysis  of  variance  was  done  on  the  time-per-frame  data  for  the  eleven  authors  who 
completed  at  least  two  lessons.  The  effei  ts  of  the  Author  and  the  Preparation  Sequence  vari- 
ables were  tested  against  the  residual  (interaction)  variance,  and  neither  was  statistically  sig- 
nificant. Also,  a plot  of  the  time-per-frame  against  the  Type  measure  (percent  procedure) 
revealed  no  apparent  relation.  Thus,  none  of  the  three  major  variables  considered  had  a de- 
tectable impact  on  the  efficiency  of  lesson  production. 
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l'he  distribution  of  the  time-per-frame  for  the  .!'*  task  lessons  is  shown  in  Fig.  2.  l'he  vari- 
ation Is  large  it  ranges  from  t to  it  hr  per  frame.1  l'he  variation  is  so  great  that  it  would  seem 
particularly  important  to  understand  the  soutves  of  difficulty  in  preparing  the  lessons.  How- 
ever. a series  of  plots  and  analyses  re.ealed  only  one  consistent  effect  10  of  li  lessons  pro- 
duced as  secoml  efforts  by  an  individual  had  less  explanatory  information  than  those  produced 
first.  IVrhaps  other  variables  not  recorded  overshadowed  those  that  were,  such  as  the  degree 
to  which  tasks  were  fully  specified  before  lesson  pre|>aration  began.  Another  possibility  is  that 
both  the  total  time  and  number  of  frames  rose  proportionately  with  increasing  difficult!  and. 
therefore,  the  ratio  time-per-frame  remained  largely  unaffected,  hi  any  event,  little  came  out 
of  the  study  to  suggest  how  to  better  manage  or  otherwise  improve  the  efftcienci  of  the  authoring 
process. 
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Fig.  i.  frequency  distribution  of  time-per-frame  for  i'>  task  lessons. 


I'lie  overall  conclusion  is  that  the  feasibility  of  subject-matter  experts  producing  task 
lessons  has  been  amply  demonstrated  under  realistic  field  conditions,  although  the  sources 
of  difficulty  in  the  authoring  process  did  not  reveal  themselves  m the  data.  Of  course,  final 
judgment  must  be  reserved  until  it  is  known  how  effective  the  lessons  are.  It  is  necessary  to 
determine  how  much  training  each  lesson  supports  m order  to  be  able  to  calculate  the  usual 
index  of  the  cost  of  producing  a lesson,  the  number  of  hours  of  authoring  per  hour  of  instruction 
delivered.  An  analy  sis  of  the  procedural  lessons  showed  that  there  are  about  -10  steps  in 
the  task  per  lesson  on  the  average.  The  tasks  especially  the  first  time  they  are  done  re- 
quire considerable  time  to  perform.  As  a result,  the  lessons  support  far  more  hours  of  in- 
struction than  might  be  expected  on  the  basis  of  previous  applications  of  computer-based  in- 
struction to  formal  technical  schooling. 

B.  Field  Test  of  the  Lessons 

A field  test  of  the  *0  lessons  is  being  conducted  at  the  sth  c'c'Gp  of  the  AFC'S  at  Kobins  Air 
Force  Base,  Georgia.  1'his  is  part  of  a larger  study  to  assess  the  cost -effectiveness  of  this 
new  technology  as  a means  of  modernizing  training  within  this  military  organization.  Hie  ma- 
terials prepared  using  the  AHFA  Authoring  System  are  delivered  on  the  l l'S-s.  1'he  perfor- 
mance will  be  compared  with  that  of  a conventionally  trained  group  one-on-one  instruction  by 
expert  technicians  - at  the  'rd  C't'Gp.  An  independent  research  company.  Scientific  Applica- 
tions, Inc.,  has  been  engaged  by  AFC'S  to  evaluate  training  at  each  site.  Since  the  capacity  of 
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the  computer-based  terminal  to  deliver  qualiti  instruction  has  been  established,  the  compar- 
ison of  pre-  and  post-test  scores  for  the  10  to  Is  trainees  of  the  experimental  group  .t  the 
kth  c'C'Gp  will  constitute  largely  a validation  of  the  lessons  themselves,  and  therefore  the  au- 
thoring system.  The  independent  study  results  will  be  reported  m the  summer  of  tv'7u  by  Sci- 
entific Applications,  Inc. 


Meanwhile,  another  kind  of  data,  the  records  matte  on  digital  cassette  tape  on  the  l.TS-5 
amts  during  training,  constitute  a source  of  lesson  validation,  summative  as  well  as  formative 
evaluation.  Consider  two  characteristics  of  the  lask-orientei  instruction  provided  m this  case. 
First,  conditions  in  the  learning  situation  are  very  much  the  same  as  those  encountered  in  the 
shop  and  field.  Second,  the  trainee  can  not  complete  the  lessons  without  performing  the  tasks 
correctly,  because  checks  at  each  stage  of  task  performance  are  an  integral  part  of  each  task 
lesson.  Therefore,  the  fact  that  a lesson  lias  been  completed  during  training  is  strong  evidence 
that  the  trainee  will  be  able  to  perform  the  task  under  work  conditions,  and  by  definition  the 
training  is  valid.  (This  prediction  is  based  on  the  usual  finding  in  psychometrics  that  a work 
sample  test  is  a reliable  predictor  of  work  performance.) 

Data  for  US  hr  of  training  have  been  obtained.  Four  trainees,  recent  electronics  graduates 
of  the  Keesler  School  for  Applied  Aerospace  Sciences,  have  each  taken  the  first  17  lessons.  Be- 
cause the  more-complex  procedures  need  rehearsal,  several  of  the  lessons  were  taken  twice. 
The  initial  results  indicate  that  it  takes  about  3 hr  to  complete  a task  lesson  on  the  first  try, 
and  half  that  time  on  the  second.  The  average  time-per-frame  is  on  the  order  of  s mm.,  a re- 
flection of  the  task-oriented  nature  of  the  lessons,  \nother  consequence  is  that,  despite  the 
large  amounts  of  time  spent  producing  a frame,  the  usual  measure  of  efficiency,  the  nut  :uer  of 
hours  of  authoring  required  to  support  1 hr  of  training  is  only  about  7 2 hr  (>er  hour,  low  in 
comparison  to  other  circumstances. 

These  initial  results  suggest  feasibility,  but  do  not  m themselves  prove  that  task  training 
in  this  manner  is  cost-effective  in  on-the-job  environments.  In  the  next  section,  a cost  analy- 
sis is  applied  to  the  case  of  the  Combat  Communications  Croups. 

C.  Cost  Analysis 

The  use  of  technical  aids  to  work -performance  training  in  the  on-the-job  environment  is  a 
new  area  of  application  of  automated  instruction.  The  conditions  are  very  different  from  those 
in  school  environments.  Thus,  an  analysis  for  the  Combat  Communications  Croups  is  presented 
here  that  may  generalize  to  other  on-the-job  environments.  It  is  based  on  the  actual  cost  in- 
curred in  the  project,  on  preliminary  results  of  the  field  trial,  and  on  estimates  of  operational 
costs  for  a 5-year  period.  The  project  budget  is  summarized  in  Table  111  in  terms  of  one-time 
and  continuing  expenditures.  These  serve  as  the  primary  source  of  data  in  the  cost  analysis. 

Consider  first  the  cost  of  delivery  of  instruction.  Delivery  units  are  amortized  over  a 
iO-year  period.  The  Air  Force  currently  has  five  machines  and  if  they  are  employed  over 
10  years  in  75  percent  of  single-shift  training,  the  total  capacity  for  training  is  75,000  hr.  The 
cost  of  five  units,  at  an  inflated  price  that  reflects  the  high  cost  of  production  in  an  R&D  en- 
vironment, is  shown  in  Table  III  as  $220,000.  or  $44,000  per  unit.  Other  expenses  include 
$50,000  for  unit  maintenance  ( $1 000  per  unit  per  year),  and  $110,000  for  training  supervision 
(10  percent  of  total  training  time.  7500  hr,  at  $15/hrl.  The  total  ($11*0.000),  over  75,000  hr 
of  training,  implies  approximately  $5/hr  as  the  cost  of  instructional  delivery. 

The  other  major  cost  is  lesson  development.  The  lessons  are  amortized  over  5 years. 
Almost  all  the  cost  is  in  labor,  8e00  hr  for  initial  versions  of  the  lessons  and  an  additional 
50  percent  or  4300  hr  for  lesson  maintenance.  Much  of  the  graphics  arts  preparation  was  done 
by  Air  Force  personnel  and  is  included  as  labor;  the  cost  of  supplies,  travel,  etc.,  was  very- 
small  in  comparison  and  has  been  neglected.  The  total.  12.400  hr.  at  $15  hr  amounts  to 
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TABLE  III 

SUMMARY  OF  PROJECT 
(SlOOO's) 

COSTS 

Cost  of  System  Development; 

Author  Facilities 

30 

Authoring  Procedures 

180 

Research 

130 

Total 

340 

Cost  of  Prepaiation  and  Delivery: 

Delivery  Units 

220 

Lesson  Prepaiation 

190 

Total 

410 

$190,000.  There  are  37,500  hr  of  training  capacity  available  in  5 years,  with  5 machines  op- 
erated 7 5 percent  of  a single  shift,  and  therefore  the  cost  of  instruction  attributable  to  lesson 
preparation  is  about  $5/hr. 

Based  on  an  exact  calculation,  the  total  cost  per  hour  is  $10.26.  This  is  32  percent  less 
than  the  $15/hr  assumed  for  the  cost  of  an  expert  who  might  serve  as  a trainer  or  as  an  author. 
An  investigation  has  been  conducted  of  the  sensitivity  of  the  cost  of  instruction  to  the  major 
parameters  that  might  change  due  to  circumstance.  Consider  three  cases.  (1)  If  the  cost  of 
production  copies  of  LTS-5's  is  one -third  that  of  the  Laboratory -built  units,  the  effect  on  cost 
per  hour  is  a decline  of  20  percent.  (2)  Doubling  the  training  capacity  from  313  to  626  over 
5 years  would  reduce  the  cost  per  hour  by  25  percent,  because  lesson  materials  make  up  half 
the  cost  and  they  would  be  amortized  over  twice  the  population.  (3)  Were  the  lessons  to  sup- 
port 80  instead  of  120  hr  of  training,  the  cost  would  rise  to  about  $1  3/hr.  The  conclusion  drawn 
from  these  investigations  is  that  no  one  variable  dominates,  and  that  there  are  savings  in  the 
cost  of  instruction  over  a wide  range  of  conditions. 

Another  potential  benefit  overshadows  the  improvement  in  the  efficiency  of  training  that 
machine -aided  instruction  permits.  It  relates  to  the  timeliness  of  instruction.  A matter  of 
paramount  interest  to  the  unit  commander  is  the  months  on  duty  the  average  trainee  spends 
before  being  fully  qualified  to  maintain  equipment.  Currently,  trainees  in  the  Combat  Com- 
munications Groups  remain  6 months  or  more  in  a training  status  before  they  become  fully 
qualified.  On  the  basis  of  the  initial  results  with  LTS-5  training  - the  best  data  available  - it 
is  estimated  that  the  total  requirement  is  160  hr  of  instruction.  Spread  over  6 months,  this 
amounts  to  6 hr  on  duty  for  each  hour  of  training.  The  rate  for  LTS-5  instruction  has  been 
about  3 hr  of  training  per  man  per  day  or  2.7  hr  on  duty  per  hour  of  training.  Given  that  120  of 
the  160  training  hours  can  be  met  on  the  LTS  units,  the  total  saving  of  time  to  reach  qualification 
is  400  hr,  40-percent  less  than  usual.  This  benefit  derives  from  the  fact  that  live  trainers  are 
often  not  available  due  to  the  press  of  other  duties,  but  LTS  units  are. 
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D.  Other  Finding!* 


In  many  respects  the  effort  reported  heri>  is  a unique  one.  the  application  of  technology  to 
hatuis-on  training  in  the  work  environment.  The  opportunity  to  compare  conventional  lalnir- 
intensive  amt  machine -based  methods  made  some  issues  clearer,  and  these  represent  advances 
in  understanding  the  nature  of  training  in  this  neglected  area.  The  broader  implications  are 
reviewed  here. 

It  is  feasible  for  experts  in  the  maintenance  and  the  operation  of  technically  sophisticated 
systems  to  prepare  lesson  materials  that  convey  their  expertise  to  novice  technicians.  Cap- 
turing the  expertise  is  accomplished  by  adherence  to  a fairly  straightforward  set  of  authoring 

procedures,  it  does  not  depend  on  assistance  from  educational  specialists  or  professional  media 

^ s <» 

experts.  Preliminary  results  in  this  regard  confirm  those  from  other  applications,  ’ namely 
that  sophistication  of  lesson  design  is  not  essential  when  training  is  delivered  by  a machine  lti.it 
uses  a computer  to  assure  that  learning  occurs.  In  the  preparation  of  task-oriented  instruction, 
the  sequence  of  steps  in  the  work  procedure  provides  a basis  for  the  author  to  organize  tin*  les- 
son and  to  distinguish  need-to-kuow  from  nice-to-know  explanatory  material. 

The  cost  of  lesson  development  for  each  hour  of  training  is  affected  greatly  by  the  size  of 
the  population  over  which  the  investment  is  amortized.  Costs  of  production  can  he  controlled 
bv 

• Avoiding  unstundardized.  untested  procedures  as  the  basis  for  a task  lesson 

• Using  an  expert  in  (lie  subject  matter  to  author 

• Simplifying  the  frame  design  with  respect  to  graphic  arts  audio  recording 
and  computer  programming  requirements 

• Using  procedure , not  training,  to  manage  the  authoring  process 

• Providing  facilities  to  check  out  lesson  program  logic 

• Making  certain  lessons  are  tried  on  several  trainees  before  filial 
commitment. 

The  general  impression  is  that  planning  and  management  are  far-more -important  factors  affect- 
ing the  efficiency  of  lesson  production  than  educational  considerations,  a good  match  to  the  man- 
power resources  available  in  on-the-job  environments. 

In  the  past,  emphasis  has  been  placed  on  two  methods  of  improving  task  performance,  pro- 
vision of  more  detailed  and  precise  instructions  in  job  aids  (manuals,  etc.!,  and  improvement 
of  the  problem  solving  skills  of  the  technician.  As  to  the  first  method,  for  complex  sy  stems 
there  are  so  many  contingencies  Unit  the  technician  faces  in  the  course  of  performing  a standard 
maintenance  procedure  that  it  is  not  feasible  to  anticipate  and  provide  for  them  all;  performance 
depends  on  comprehension  of  the  nature  of  the  equipment  and  the  task.  \s  to  improving  reason- 
ing skills,  the  journeyman  technician  only  occasionally  is  required  to  find  faults  and  solve  other 
problems  by  inferential  methods.  Training  is  directed  mainly  to  learning  the  equipment  design, 
operational  doctrine,  safety  matters,  and  work  steps,  not  as  separate  skills  but  as  they  apply 
to  execution  of  procedures  to  achieve  specific  work  goals.  No  amount  of  prior  component  skills 
training  nor  improved  documentation  cun  substitute  for  tins  kind  of  one -on -one  instruction. 

In  the  area  of  task  training  for  complex  equipment  and  sy  stems,  it  is  quite  clear  that  the 
usual  "ISO"  (Instructional  Systems  Development!  procedures  have  limited  application.  Die 
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principles  on  which  the  procedures  rest,  such  as  those  given  in  the  Intcrscrvtcc  Manual  ' 
sound  but  the  methods  are  not  appropriate,  t here  is  an  mupialified  recommendation  to  analyse 
the  work  tasks  into  component  skills  and  to  develop  separate  training  for  each  area.  This  ap- 
proach by  itself  is  inadequate  as  preparation  to  perform  complex  tasks,  when  the  criterion  oh- 
jective  is  to  have  the  trainee  demonstrate  proficiency  m the  task  itself. 

K.  Summary 

The  resources  required  to  prepare  and  conduct  technology  -based  performance  training  m 
on-the-job  environments  are: 

• The  AHPA  Authoring  and  Management  Procedures 

• Subject-matter  experts  to  prepare  materials 

• A small  computer  system  to  record  audio  and  logic  and  to  analyse 
data 

• Stand-alone  training  units 

• Operational  equipment  to  train  on. 

The  preliminary  results  of  the  initial  operational  test  and  evaluation  of  applying  these  resources 
in  the  Combat  Communications  Croups  indicate  that  at  least  JO, 000  hr  of  highly  qualified  tech 
niciau  time  and  t 25,000  hr  of  recently  qualified  technician  time  can  be  saved  over  the  next 
s years.  In  addition  to  the  investment  of  available  resources,  outside  procurement  fund  ■ re- 
quired to  realize  this  benefit  have  been  on  the  order  of  10  percent  of  the  value  of  the  projected 
labor  savings. 

This  means  for  improving  the  efficiency  of  task  training  can  be  replicated,  and  is  believed 
to  have  widespread  application  in  the  Armed  Forces.  Quantity  production  of  I TS-1'  units  yy  ill 
reduce  the  cash  expenditure  required  to  less  than  s percent  of  the  value  returned. 
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